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ABSTRACT 

The Rasch model for test analysis is a latent-trait 
model, which specifies the relationship between observable test 
performance and the unobservable traits or abilities assumed under 
test performance. In most cases, the test constructor has no clue as 
to whether the latent traits postulated by the model are indeed the 
abilities he wants to measure. A study using an English-language 
listening comprehension test for both native speakers and Dutch 
students of English as a second language has yielded data that may 
help to identify the ability that, according to Rasch test analysis, 
underlies the main trait measured by the test. (Author/MSE) 
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Poeuslng in on a Latent Trait i 
An Attupt at Construct Validation by 
Mans of tha Raach Models 
John H«A«L* da Jong 



Thm Jtaach aodai for tm»t anaiyaia la a ao-callad 
latant trait JWdai. In a aodai of thlm kiad, a 
ralatiooahip ia mpoelfimd tetvaan otearvaJbia teat 
par/oraunoa and thm mobsorysblm traita or abilities 
mmsumed to mdwlie par/onunca on tha taat. 
In aoat GA9m» thm taat cooatructor haa no ciua as to 
irbatliar tha latant tvalts poatulatad by tha aodai axa 
indomd tha ahiiitiaa ha wanta to Maauxa. rrjrout 
rnmmmlons of a iiataning co^paahanaion taat cn nativa 

•a' on foroign iaaguaya iaaroara hava yiaidad 
dkta poaaihiy otfmrlng a aaaaa to idontlfy the ability 
Mhich, according to Jtaacb taat Mnalymim^ underlima tha 
Min trait Maaurad ^y tha taat. 



Introduction 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



Aa tha aubtitla of ay papar indicataa 1 will try to prove 
conatruct validity of a taat isy Mana of tha Raadh aodel. 
Tha taat I will deal with ia a foreign language liatenlng 
co^prehenaion taat, whidh haa recently been developed at 
our Znatitute for Xduoational Meaauraaent (Clto). I will 
atart with a brief deacription of the teat. Secondly I 
will define the abili^, which we propoae to aeaaure with 
the teat, which la in fact the theoretical conatruct. 
In the bo^ of the paper I will offer evidence for a 
relationahip between the conatruct aa defined and the 
■eaaura of fit of itena according to the Raadi nodal. 
Finally I will illuatrate the aaaumptiona nade in this 
paper with examplea froa the actual teat. 
After Ihiel Theuniaaen^a paper (thia volume) I feel free 
to aaaune a certain knowledge of the Kasch nodal but I 
will ej^laln oonoepta of particular laportance for the 
evidence which I hope to give. 



The teat itaelf 
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Thrs document has been reproduced as 
received from the person or organization 
originatlrig It 

□ Minor changes have been made to improve 
reprcduction quality. 

• Points of view or opinior>a stated In this docu- 
ment Oo not necessarily represent official 
OERI position or policy. 



It ia important to nake the prellninaxy renark that the 
teat waa developed aa a reaearch projects to inveatlgate 
tech niquea of teating foreifp language liatenlng 
oonprehenaion# tachnlquea which were new to ua* Obviaomly 
audi a teat will contain a larger nuabar of iteos of poor 
quality than any of our teata oonatructed for actual uae 
in final exanlnationa* 

The teat uaea life recordinga taken fron different xadlo 
programa, cut into rather abort aamplea of f^>*:)ken 
language (about 25 aaconda each). The language to be 
teated la Bngliah. Teataea Uaten to the tape, hear each 
aa^ple only once and have to reapond to a nultiple choice 
queation with two qptlona printed in a teat booklet. They 
have a 10 aacond pauae in between aaiq>lea to decide on 
their diolce* 
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two tgrptts of itMM war* usadt 
A yipoo-falsa It— 

riS'IIL!!?!! ?* •t«tam.nt in tha 

taat booklat la in aeoordanca with what haa baan aaid 
on tna tapa* 
■ Wodifiad e loaa Itaaa yjth two optiona 

UlTe*^ ?• <>«latad fro- tha ta!* wara not randoady 
aalaetad but wara <dio(ian for thair aaa«ntie ralavanea. 
in «aeh aa^ia a word or ^roup of worda waa cut out 
fro« tha tapa and raplaead an aleetronieal sound. 
Taataaa «ar« to daeida ««leh of tha two option* 
praaantad in thair taat beoklat could ba usad to 
raator* tha taxt. Thus tha typical pr^blaa with eloaa 

i!'^7-!S°*'lS***** "^'^ soaring - could 

ba avoidad and no productiva akill was ^tad. 

Tha taat waa triad out on two groupat 

- a rapraaanta^iva sa^pla of 575 Dutch piqtlla takan 
fro. tha targ«»t population (pu?ila praparing for 
final Sacondary vSchool axaadnationa af tar 6 yaara of 
Bngliah as a foreign languaga) 

- a group of 30 nativa apaakara of ooat>arabla aga and 
aducational background 



Tha conatruct 

^•"*»tion it ia nacaaaary to datamdna tha 
concept which accounta for parfomanca on tha taat 
(African Paychological Aasociation, 1974| Cronbach and 
Naahi, 1955). In a for<i>lgn languaga liataning 
coa^rahonaion taat wa -opa that dlf faranca in raw aeoraa 
on tha taat can ba aivlainad hr diffaranca in ability to 
undaratand tha apokan languaga in question. 
What ia the ability to undaratand a apokan language? 
Without going into the discuaaion on the Unitary 
Coi^tenae Hypotheaic varaua the belief that Unguiatic 
competence can be broken down into a number of totally 
diatinct factora or Vollnar*a auggeatioa of a 
hierarchical nodal (Voll^r and Sang, 1981i Vollaar, 
1981) one can aafely aaauM that native apaakara of a 
languaga do poaaass thia ablUty. 

^iJJ.^'^ ""^^^ apaakara differ in their 

? l*n9a*g« according to their 

ability to oqpe with the languaga MterUl at the 
conceptual level. 

Obvioualy, defining foreign language Uatenlng 
coivrahansion aa 'the ability native apaakara deaonatrate 
in underatanding apokan aa>9laa frw their native tongue* 
la too coiBprehensive. Tb rule out non-linguistic factora 
the concept ahould be narrowed down tot 'The ability to 
undaratand the foreign language at the level of native 
apaakara of comparable age and educational background*. A 
group of native apaakara thua defined will have to do 

!J^*I?Jf ^'•^ ^ in «y caae, 

JJ^>.riL r ^f^^ the i^at able liaten<,ra 

in the target population of non-nativea. Furtharaore no 
aignif icant variance in native apeaker aeorea on tha teat 
ia to ba expected. I will not go into thia poiS 2^ St 
saall variance in native apeaker aeorea on the teat has 
been reported (de Jong en van dan Mieuwenhof, 1982). 
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Xii tans ioC $i»m Bodttlt tH« aodel poatuUtas 

anidlMMloMUiy of tlM ability or undarlying trait. 
Bacauaa na axpaet natisra apaakara to ahow graatar abiUty 
in liataning eoi«r<^nai4»ii, thay ahould haw higher 
probabiUty of gatting tha right anawar on aach itan, if 
tha itaa raqulraa liat«i»ing ability. 




lloK plaaaa eonaidar figura 1 whidt rapraaenta the iten 
chaifaetariatie eurva for an iuginary itaa i. Diffarant 
lavala of ability (s^xia) ara plotted against the chance 
of gatUng item i right (y-asia). A person of abiUty 'a* 
a.g. haa exactly .50 chance to get the iten right. A 
dianca of 1 .0 ia of oourae the higheat poastible cdiance 
and would require ability at ♦ in other words a chance 
of 1.0 ia the upper Uait, the oeiUng for any person 
doing itaa 1. 




^^^'^•^'•P'***^^ thia reUtion between ability and 
probabiUty of getUng itea i right again. On the y-axia 
a dotted Una at .8 repreaenta the obaervad proportion of 
right anawara in a group of native apaakara defined as 
before. Tvom their observed reaponaea «e can astinate the 
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«Mn tfblUty oC thl« vroap of Mti^ spoakont m^^. 
And teeauM w #xpaet noa-nativM to shoir loss ability in 
solving Uio Itom than nativo spoakors • thoir ability 
lovol on the x-axis should ba to tha laf t of that of 
nat&va gpaakara (a^) • va axpaet thaa to hava lass 
Ghmem in gatting tha itas right. This iaplias that for 
tha t#rgat group of non^nativa apaakara tha uppar liait 
of thair thanea to gat tha itaa right ia aat by tha 
native apaakar aooras on tiiat itaai for thia itaa (and 
not no)« Zf thay acora highar than tha nativa apaakara 
than tha abiUty raguirad for tha itaa ia not tha ability 
wa aat out to saaaura* 

For thia raaaarch Raaeh analyais vaa dona by coaputar 
with tha program CALPZT (Hright and Noad, 197S}« Tha 
unconditional Msdmia likalihood procadura (UCON) of thia 
program was usad to aatlMta al ^Uty and difficulty 
paraMtara (Wright and Stona, 1979 h 
Tha program divldaa taataaa into aix groupa of roughly 
tha aama aisa according to laval of ability aatimatad 
from thair parformanca on tha taat« 
Tha program than caleulataat for a vary itam^ tha 
probability of auccaaa on that itam for tha aix groupa, 
and axpraaaaa tha probabiUty as an as^^etad nunbar of 
right anairara in aach group* 
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In fact thia praaanta ua with a rough aatimation of part 
of tha itam charactariatic curva (figura 3)« Only a 
'rough aatimation '# bacauaa no. mora than aix pointa 
(i.a.t at tha aix ahiUty lavala) ara dafinad, and mly a 
'part' bacauaa ability ia not likaly to vary from minua 
infinity to plua infinity among tha taataaa* 
Thia aatimaticn of tha probability of auccaaa ia comparad 
to tha actually obaarvad luwan parformanca of aach of tha 
aix groups on tha itam* 

Tha diffaranca batwaan tha aatimatad and tha obaarvad 
parforman"^ ia tha baaia for tha calculation of a maasura 
of fit of tha itam in tha taat* 

Of course thara ara numerous possibilitiaa for departure 
of the obaerved performance from the ea time tad curve* 
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^ mt th9 •bitlty Uviii Of ft 9V0UM. ^ 
Oil Itco ii vook dlocrlBiootloR botwvon areupo. 
M itOB 3, otvoof tfiocviamotioii botwooo vroupo. 



two iteu (i and j) of equal dlfficultyi the probability 
<rf •ucccss for oaeh of the sis group, i. the saae on both 
iten. The iteu do not differ mch in degree of 

«rfril ^ •«P«°«*0 «>'v» thu. their .easure of 

fit in the teat will not differ mch. But on iten i a 
Urger nuaber of persona froa the low ability groupe gave 

•«P«°**<>» in the highS 

abiUty group, there are fewer right anawera than 

*2Tf?^' <>i«cri«lnatea less well between the 

ability group, than we would expect froa their 
perforaance on the whole test. For Itea j it is just the 
opposite! this Itea is highly dlseriainative. 
in fact, the difference in ability between the aix groups 

the adalnlatxaUon of the test, thus the differences in 
SrriJ!;'? ^?*!" constant on 

siiiSr 

How in postulating unidiaenaionality of the ability, what 
r/T*^ in y«n«r«l is that whenever an itea doea not 
itU*il*Li^' «»e aaae dlfferencea In 

"9«>i'«» • different 
SS^SS r^il "^''^ neceaaarily .how high scores 

However, iaagine a teat constructor designing a test to 

■ore.tMngtt to haaaer down than ««ii, thin oneTdo. He 
JSrLS tTS?;: taking hi. te.t by the nuaber of blow, 
ttey need to drive la each nail coapletely. Obvioualy. if 
he preeenf hi. teet to people who are poJr at TJlnll H 



!• UabU to find phyaieal stMngth to to hlgtor 
corMUtod with numtor of sMahad fingtrs than with 
nuator of blows. Mut ho ia doing ia aeaauring two 
diatinct traito or abilitioa at tho aama tiaai phyaical 
atrangth and akin at aiaing. 




Pigura 5 givto a graphic rapraaantatioa of aueh a teat. 
T»at T rwiuiraa two diatinct abilitiaa A and B in a 3i2 
ratio. Hhm itoaa requiring aithar ability A or ability B 
exclaaivaly an vapraaantad ca doto on two linaa 
interaacting at an angla of 90*. XteM at tha 
intoraaction of tha two linaa raquira no apacific 
abiUty, naithar for abiUly A nor for abiUty B. Moving 
away froa tha intoraaction, ito&j raquira aora and aora 
apacifidy aithar ability h or ability B and will 
conaaquantly dlacriainato aora oa that ability totwaan 
paraona taking tha toat. Bacauaa tha toat raquiraa both 
abilitiaa a and B, tha latont trait of toat T will to 
dafinad aa a lim going throu^ tha intoraaction of trait 
A and trait B at an angla dafinad by tha ratio of tha two 
abilitiaa in tha taat. For itoaa to fit in tha toat they 
will iviva to toat both abiUtiaa aora or laaa in thia 
ratio or ona of aithar ability at a not too apacific 
level. Itoaa diaeriaiaating highly on either ability %rlll 
not fit. 

Beeauaa toat T contains aora itoaa requiring ability i& 
than itoaa requiring abiUty B, highly diacriainaUng 
itoaa OB tha trait of taat T are aora likely to toat 
abiUtgr A apaeifidy, whereas itaas with low 
dlaeriaiaaUva power in tha toat have a graator dianca to 
to itaas testing abiUtr B. 

in the axaapla of tha taat oonatructor who wanted to toat 
pIrysieaX strength the best solution for hia ia to get rid 



of lh« any, thin nalla that rcquir* a good aim. 
What will happan with tha taat la ahown In figure 6. 




o 



It«M raquirint akility * •Bcluaiv.W. 
ZtH* ,.quiriii« akiaty • •>elu«i»«ly. 

nl'llll |f.**"» koth rtiuty » 

latont trait of i««t T'. 
tiait* for it»« to fit in teat T'. 



Teat T' atlll raqulraa the aane two dlatinct abllttlea. 

***r.J^" T and a new 

definition of tha Utant trait for teat T- 1. obtained. 

angl. for«id by tha latent trait of teat T' and 
•blUty A haa bacoM aaaller and eutoaatlcally aore lte» 

^acrl^natlva powar on ablUty A now fall within the 
Ualta for iteaa to fit in teat T». 

Wth thia plctora in mind we can redefine cur expectlon 
Mntloned aarUar. whenever an ita. doea not fi^IJlS! 
S!!!?^?' coaprahenalon teat and thia Ite. haa a low 

do wall on thia itaa. In fact, they night juat aa well 

'fHHJ^'."^ »on „a Jvea f ro. Ihr 

target population, if, howavar, an Iten doea not fit .nd 

S'Sa ^^"'^ «>i.cri.lnativ. powJr^^Sa'^iir' 

?: J^^" "ill eventually fit after the 

iten. ahowing i^.fit i„ coabination with a low 
dlacrlnlnation Index will have been deleted fco. the 

StSSn'nnf!!' r^^"^ *** • correapondence 

Sinili^^^r ■'^^^^y ^ l**«nt trait 

obaarvad In Reach analyala of foreign language learner 

Saarild. "■^'^ •P^'**' 
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In flgura 7 th* dl«crl»lM«.i 4 ^ 

horUonUl Uno r.proM«t^ S!-****, »»• ««ottod 

«»• vertical Un. r.p^!„j! Jjl^ °' froodo.). 

«>Mi tho tost ia oonorSMl^^ • *^«°'i«in«tlvo power 




•pookor group m» thiuL^l^l T 
1« g.n.r.1 highly ^.cSTnJS^ SSttJni'i?'*" 
conprrtiwwioi, tMt. ^ Ustonlng 



Er|c 9 



bMHdinlMto of 
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In fi9ur« SB avvry dot r«pr«8«nta one of th« 59 iteu in 
ttim Uatening coaprvhansion teat, itie proportion of 
natly* apeakera who gave right aaawera on each ite« (x- 
•xia) ia plotted agalnat the aeaaure of fit In Reach 
•nelyaia of the reapbnaea obaerved in the target 
popttUtion (y.^a). vhe critical apper liait of the fit 

ril!?'*/^'''' ^' *V • horiaontal line. The 

vertical line repreaenta what we take aa the lower liait 
"P^aker reaponae. Hoat of the iteaa (35 out of 
59) fall within both liaita. Another 9 itema aurpaea both 
Uaita and thua confira the hypotheaia toot alafitting 
iteaa are too difficult for natives becauae they aeaaure 
Oie wrong trait, they do not teat native apeaker 
llatenlng ablUty. a total of 44 (35+9) iteaa coae up to 

alafit that are not too difficult for natlvaa, and 7 
iteaa with a low naUve apeaker acore apparently fitUng 
in the teati a total of 15 iteaa contradicting the 
hypotheaia. Pigore 8A preaenta the aaae findlnga once 

10 _ 



rSi^°irif*i**" — "" o' and native .oeak.r 

•eora M diehotoM variablM is .37. ""^iv* speaker 

«n pnsttntM in figures 9A and 9B. 

"S5n1:t2 L'S.'^ S^L"""^ °~ ~~ 
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Another 3 lt«M an b« del«Ud fro. th« teat nw (figure, 
m and 1 0B) leaving 47 itmm In th. f .t. t«, max. Td^ 
are now within both Ualta whoroaa 3 nora aurpaas both 
lialtai a gain of 5 if mm aupportlng the hypotheals. Only 
6 ItoM are left that ao not eorreapond with our 
ejqpeotatlona. Phl-oorrelatloa la now .77, 
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Drt«tlng th« Uat thn* it«M that aucMs. both ii-i*. 

U«lta and «.rilii!? ' **** Wiiias. both 

conSdictlM f hypoth«.i. .nd 5 

4wJlSrii^tii: Jj*r*«-1*<^«» i« .so. Bat of th« 
-„nn:^ •PP«r«iitly alsfit in apit* of 

*^,?!! *^ population, tha twjat 

population did battar than natl^ apaaXara on thilJ'SJo 
itana. Thla aaana that tiia aiafit of thaaa itana in flet 
doaa corraapond with oar a^taUon. 



• 11Aan<1ia:«MM 

ki 
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FigoTM 12A and 12B ihow what hafipms if HmUtm tiiesa 
two IteMs OM worm itM falls within both Units and 
only two oat of tha initial 59 itana rafusa to oom up to 
our ajqpactations ona aUghtly aiafitting itaa aasy anough 
for nativaa and ona itaa that ia too difficult for 
nativaa but aUU fita. tha 40 itaaa falUng within both 
linita now eluatar in tha bottom righthand oornar of tha 
graph in figura 12Bs thaaa itaaa fit axtraaaly well in 
tha taat and praaant no dif ficultiaa for tha group of 
nativa apaakara* 

Corraapondanca batwaan nativa apaakar ability and tha 
latant trait obaarvad in Raadi analyi^ia of tha targat 
population raaponaa axpraaaad in phi-corralation ia .9^, 
and confiraa our hypothaaia that tha latant trait of this 
taat tm in fact tha ability of nativa apaaJcara to 
undaratand aaaplaa of thair aothar tongua* niua 40 itaaa 
out of tha original 59 itaaa will aaka a valid taat for 
■aaauring liataning coaprahanaion of English* 



Asm 12 A 




iooaffiaa far 



(PIS) 



r 



221 
2 



• . 

• ••• 



«1 



2 :$ A 

proportion of imIm oi^ 
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Th« MMinlng 19 itMw do not Maaar* this ability* 
Figaros 13K and 13B roproaont tho result* of a Raaeh 
analysis of thesa 19 itOM ragardad as a separata test. 
Pigura ^3^ pietarea tha itsM in thU teatt 15 itaaa 
aorpaaaing both critical Uaita and 4 iteaa m^rpaaaing 
on^ ona of thaaa Uaitst praaantad aa a aaetion f roa tha 
orginial 59 itea taat (on tha laft) and regarded aa a 
aaparato taat (tha eirel* oa tha right). Figure 13B ahowa 
that (apart froa ona aUghty aisfitUng itea) theaa iteaa 
viU fit together in a teat. Nhat this teat aeaaurea is 
not clear aa yet but anyhow it doea not diacriainate 
betoieen abla nnd leaa able llatenera of Engliah of 
co^rable age and educational background. 



n»MnA«idiia 




(PSJ> 



F«0Maa 



221 
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X would nov likm to illustrato the different kinds of 
items in the test by giving examples of items that do fit 
in the test and of those that do not fit. 

Itie first example is item number 17« 



Itm no 17 




Tap«i 




(vllMnt) 


Ona of tha things that norriaa mm about thaa 
ia how paopla ara going to ba abla to kaap 
track of thair apanding whan thay*«i joat 
got a plaatic oard which thay hand to tha 
ahop kaapar* sow*** 


9>Mtlon, that if yoa'n ptyiag for 9ood« 
vith that md of aotm yon ean wm how Miqr 
yoo*«i sptBt mud TOO havo mm idoa oC tho 
total of your 4ay*s sheppiiig «haMaa if yoo 
«o into a onabar of ah ahopa and pay with A 
tho 8«M eradit card, yoo com avay vith 
nothing ethar than racaipta and thara*a no 
aanaa of actaally apanding aonay. 


(^laatiQii booklatt 


17 Tha aaa 

of tha 4 
a Tnaa 


of a cradit card can aaka paopla laaa awara 
BMoat of aonay thay apand* 
a palaa 



thm proportion of th« native apeaXer group that chose the 
right answer (A) was 1 .00 and .86 of the target 
population dioee this answer, hll the native speakers 
found the right answer obvious and for the target 
population it was an easy itea. The item is easy if you 
understand English. Nevertheless, the itea iniUalTy 
showed aisfit (3.86) in Rasch analysis of the target 
popuUtion response. Misfit is ooabined with a high 
discriaination index of the itea (2.35) but disappeara 
after the operation of deleting items showing aisfit 
along with a low native speaker score. Fit aeasure for 
itea 17 improved and went down froa an initial 3.86 to 
2,97, 2,47, 2.10 respectively* as aore and aore iteas 
were deleted, ia the way I have just described* ending up 
at 1.73 in the final 40 itea 'good* test of figures 12A 
and 12B. 

Figure 14 pictures the itea characteristic curves of itea 
17 in the 59 itea and in the 40 itea test. 
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lh« eorr«et umwt is A. A porposrtion of .68 of the 
nativa epeakar groop and .69 of tha targat population 
chosa this answar. 

Though .69 of tha targat populaUon indicatas th*t tha 
itaa is of nonal difficulty tha native speaker score is 
far too low. iha aajor part of the target population went 
•through tha ceiling* scoring higher than the native 
speakers. Miat is wrong? Mr. MLlson asks whether people 
are 'reluctant' to oaa tha new facilities like credit 
cards etc. Httts's answar saaas to be' 'yas' because he 
givas an aiqplanation aaying 'credit is an aaotive tern'. 
Linking this to 'reluctance' used in the question, the 
listener can conclude that people do not like the idea of 
credit. The itea is acre a test of intellectual 
gywtastics than a straightforward test of Qiglish which 
expUins why it U equally difficult for native speakers 
and for tha Urgat population lAo have a similar 
educational background. The itea, however, fits in the 59 

bttt has a very low discriaination index 
C.39}. After deleting the first 9 items in the operation 
described, fit aeasura leapa up to 2.82 and tha itaa no 
longer fita. 

Figure 15 givas tha itaa diaracteristic curvea of this 
itaa in tha 59 itaa test (figure 15A), after deletion of 
9 iteas (figure 15B) and in the 19 itea bad test (figure 
1 5C)« 
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Both tta^iM wra tnuufalsa Iteu. ^t^^ 43 1^ ^ 

■odlflad dosa Itn with tuo opUoMt * 



XUb no 43 
Tapoi 



mac* CterlM hM bMa «pc«)tioy ^boitt BritAlA** 
ladMtrlAl ftitttn. told di^v^tM at a eonforvo¥ U 
•omMMttth of tho iron and ntimik txmOm onion tlub it 
only tho tvo sUno off ifitentry vooXd eo-opor«to 
Britnla'n porfocnnneo intornartonnlXy cenld bo -^IM%-^ 



Omotloa booUoti 

43 K footly i npc ofd 
• vocy diMppointlo9 



I i ! '-OO ti* natlv. and 

.86 of th« tar9«t popoUtlon ehoa* tiM right «/><w%r (A), 
It 1« «n aaay lte> bat not tod «My. itev«rth^W<». th« 
Ita. do., not fit (5.41) .» di.cri.ln.tion ij ^^i^'hSh 
(2.08). in th. .ub.«iu.nt st$9<ft« of deletion fit 

good fit I Th. di.cri.in.tion i«d.x i. now i.^j, pt^J. ig 
40^^?" in til. 99 *nd In th. 



npMli:liMi cteacivMB curm for iM f«4a 



100> 



flt:S.41 fn.jg 



of • iMcy 901^ ^ tf« t»0M papiMon vMv ri^ , 
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110 coiitliio# f«ith itm^ so 441 



rrliie% Cli«rl«0 m^pm^lmA tot MiiA9Miit «fid anloiv to 
rMch 4 l»tt%r miters tAMlng and ««iit «d to •sprMs hU 



Mam this itta no probiaa for thm naUve «p«ak€rtf 
1.00 of tht# group gftvo th« right answer against ^71 of 
tha targat poPalaUon* 

Initial nlafit of tha itaa in tha 59 itan tast 1« 3.31 
but goes donn to 1.73 after the daietion operation. 
Figure 17 givaa the it^e characteristic curves^ 
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aa^Jl.43 
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Xt«a no 6 la another •xa.mflm of « trlekey true-f 



Itm ae C 
Tapat 

MllMnt) ScyMir roCMacM, U this •velatloa to th* 
•UaliMtloA ct MMy* OM which ia happanlM 
vlrtoally «U e«ar tha world* 

(rortaaeoat ) Taa z thirst it it. raihapa eha aoat 

iatacastim^ plaea U Japaa, baeanaa tha 
iadivlAMl doasa'c ma chanaa in Japan, 
thay oaa cMb ami tbay oaa paaa-boeka. Tea 
can aaa Japan naliiny tha awitdi (rai eaah to 
aUetxeniQ foate tranafar witheat paaainy 
throoffi th« latatMdUta ataqa of ehaqoaa. 



giaation beeklatt 

t Japan haa alr*.i^ awitehwl trm ehaqoaa to alaetconie 
■oaay tranoter. 
* Traa ■ ralsa 



Obvloualy, naUw sp«ak«r« had ^reat trouble In apdttlng 
tbm Ineorractnaaa of tha atataMntt only .56 cho8« ansvar 
B. Of tha targat populaUoit a proportion of .52 cSid«* 
thia answar. Tha <iuaatlon la too dlfflculti "A* 
Intarpratatlon proeaaa naeaaaary to arrlva at tha ^^ht 
«nai«ar la l^poaalbla In tha ahort tiae allowad by tha 
•pead of tha ongoing tapv*' Tha tona of volca la po^lUvai 
thata la •aUsinatlon of aonay* In Japan, Japan la ahead 
in Mdamlslng tha payMnt ayataaa but juat bacaua# Japan 
la ao far ahaad and aklppad tha ataga of chaquaa 
altogathar tha anawar ahouid ba 'Palaa*. Tha Itaa phoira 
««vara alaflt (5.88) and doaa not dlacrlalnata at 
according to expaetatlon aaong tha taataaa In tha t»rgat 
popuiaUon (-.04)1 in tha 19 Itaa -bad* taat, howavvc, 
tha Itaa ahowa parfaet fit (.57) and nornal 
dtacrlalnatlon (1.02). It saaaa to na that tha 19 itaa 
•bad* taat would ba a good taat of IntalUganca or 
alartnaaa but It la not taally a propar taat of English 
Ilatanlng coaprahanalon. 

Ftgura 18 ahowa tha Itaa <!haractarlatlc curvea of thla 
Itaa In tha 59 Itaa taat «nd In tha 19 Itaa 'bad* tast. 
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A:h cuiMiii Mton M trh tt'ted*' Mm m 



2 3 4 S • 

I Miy Iv ym^iD t Of ttv 



mis? 




S: ' T^*** ^ of tarn 

tiro with a high native speaker score but an avan higher 
•core in the target population, the last two iteM to be 
deleted* °^ 



Itm no 45 
Tapot 



OgaadA'o now inmldont. Nr. Qodirmf Bom1o«, hu «p(««Ud 
to tho poopXo of tho eoaotry to oo-oporoto la buildiiM « 
RTOoporoito ooyntxy that roopocto tho ruXo of low, tho 
protoctioo of faueon ciffht* and tho ootobllalMoiit 



(Motioo bookXott 

45 A of o aov ototo 

1 of dMoenoy 



Matlv. •p.akara seoMd .ufff ici.ntly high on thia iteat 
.84, but of th« targat population .90 chose answer B. Ona 
could defend both options, but the word •deaocracy* seeas 
^ S •'••P«cta the rule of and .huZan 

ii??!? ?"? ^" '"^^ O'i^inal word. KnowUdge of 
poUtlcal jargon In a revolutionary context and thus 

'"^"^^w"'*^"'*'*** knowledge of the world is tasted 

"•^•"i"^ -^ility. the ite. fit. in 
the original 59 itea teat but after a.veral deletion 
operationa it acquire, c fit Maaure of 3.43 and i. 
deleted in its turn, in the 19 itea 'bad* teat itea 45 
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Pi9ur« 20 prttsants th« mmmn of all it*. ^».., * . 



A:Oi H M Horns 



I 
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■ Ob 40 -boot' 



of ttio lariM 



C:On «'bod* 



' 2 J 



.63 
St 

.06 



of tfli 
of 6 
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Rwi this pietnr* it is elMr that tha natlva ....v.. 

Popolation. Tha aalactlon antitlad UO baat itL-« 
craataa a largar diataaea batwaan thi ! 
tha avaraga of thTtaraH n^SlJ*!?^ «<> 
w??. *lacrlirtiiataa battar batwawi t->Jj»4MM 

point rMftina to , 

9rcupa m tha .•-b.^i^tiisrcS':::!^':: 'jr. j^'jjli ^ 

•vaxy aaparata aaalyaiat tha taataaTar.^? f^! ^' 
groapa aeoordlag to tiiair Joom ITIkI^ i"*® -l* 

ia diffarant altooathar ^^4.^^ -T^ **** 

40 ita. .baat . uSiSTtTSif 'LS^!? 

JSJaSlS^a!^'?? -l-ctiona wlthoot 

ooiVari^. to of -qual langth for a.ay 

(n-S75). ll«taiiing coi^rahanaion taat 





All 


A 


B 


C 


All 59 itaw 
At 19 •bad* itaav 
Bt 19 *baat* itav 
Ct 19 •baat* itaM 


X 

.40 

.78 
.75 


'X 
.06 
-.01 


X 
.47 


X 
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~ "l-^i^n »»tv-n tha 19 'bad" it«I and 
aithar of tha t«o aalactiona of •baat* ita« -J 

Cendmion 

!^1^2"JaJr2n*l''!^ "•**»^»» coo^«hanaio« taat 
•naiysad hara can ba idantifiad aa tha abilifcv nmJtl^ 

!TS*^T" '« taating an ability^i^ 



t"t oomtructor, to aluiSl «, th. 
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